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Abstract 

The paper presents a simple derivation of the Gustafson-Barsis 
law from the Amdahl's law. In the computer literature these two 
laws describing the speedup limits of parallel applications are derived 
separately. It is shown, that treating the time of the execution of 
the sequential part of the application as a constant, in few lines the 
Gustafson-Barsis law can be obtained from the Amdahl's law and that 
the popular claim, that Gustafson-Barsis law overthrows Amdahl's law 
is a mistake. 
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1 Introduction 

The Amdahl's law formulated about four decades ago [I] is considered to be 
one of the most influential concepts in parallel and distributed processing 
[7]. It describes the limits on the speedup obtained owing to the execution of 
the application on the parallel machine with relation to the single-processor, 
sequential machine. More precisely, Amdahl's law says, that the speedup of 



an application obtained owing to the execution on the parallel machine can- 
not be greater that the reciprocal of the sequential fraction of the program. 
Speedup restrictions resulting from Amdahl's law prevented designers from 
exploiting parallelism for many years, being a nuisance to vendors of parallel 
computers [4|. The rescue came from Sandia Labs. On the basis of some 
experiments, Gustafson [2j claimed that "the assumptions underlying Am- 
dahl's 1967 argument are inappropriate for the current approach to massive 
ensemble parallelism". Furthermore, Gustafson formulated "an alternative 
to Amdahl's law suggested by E. Barsis at Sandia". The so-called Gustafson- 
Barsis law is said to vindicate the use of massively parallel processing [5], [6]. 
However, in the author's opinion, when we analyze deeper both laws, we 
will see, that Gustafson's results do not refute the Amdahl's law, and the 
Gustafson-Barsis law can be directly derived from the Amdahl's law. 

2 Amdahl's and Gustafson-Barsis laws in the 
original form 

Although in the original Amdahl's paper [Ij there were no equations, basing 
on the verbal description one may present his concept formally. The way of 
our presentation is similar to that of [3], [4], with only one difference, which 
will be explained later on. It is assumed in the model, that the program 
consists of two parts: sequential and parallel. While the time of the execu- 
tion of the sequential part for a given size n is the same on all machines, 
independently of the number of processors p, the parallel part is perfectly 
scalable, that is, the time of its execution on a machine with p processors 
is one p-th of the time of the execution on the machine with one processor. 
Let us denote by (3{n,p) the sequential fraction of the total real-time T{n,p) 
of the execution of the program on a machine with p processors (the men- 
tioned difference introduced here is treating both the fraction (3 and time T 
as functions of n and p; it will prove to be very useful afterwards). 

With this notation we may calculate the sequential part time Tg for the 
given problem size n from the expression 

Ts{n)=(3{n,l)-T{n,l) (1) 

and the parallel part time Tp, which is dependent on the problem size n and 
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the number of processors p, from the expression 

_ (l- g(-U))-T(n.l) 

P 

If we ignore communication costs and overhead costs associated with oper- 
ating system functions, such as process creation, memory management, etc. 
[4], the total time T{n,p) will be the sum of sequential and parallel part time, 
that is 

T{n,p) = T,(n) + T,{n,p) = P{n, 1) ■ T{n, 1) + lijl^^"' ' - 



P 

1-P{n,l) 



Tin, 1) (3) 



P 

From ([3]) we get directly the formula for the speedup S{n,p) obtained due 
to the parallelization of the application: 

The formula dH) is called Amdahl's law. It is seen, that in the limit 

S{n,p) / (5) 

It means, that even when we use infinitely many parallel processors, we 
cannot accelerate the calculations more than the reciprocal of the sequential 
fraction of the execution time of the program on a sequential machine. That 
is, for example, when this factor equals |, the program can be accelerated 
at most twice, when ^ - ten times! Speedup restrictions resulting from 
Amdahl's law prevented designers from exploiting parallelism for many years, 
being a problem to vendors of parallel computers |4|. 

The help came from Sandia Labs. In some experiments described by 
Gustafson [2] it was taken, that the run time was constant, while the problem 
size scaled with the number of processors. More precisely, the time of the 
sequential part was independent, while the work to be done in parallel varied 
linearly with the number of processors. Since the time of the execution in 
Gustafson's paper ^ was normalized to 1, that is 

Ts{n) + Tp{n,p) = 1 (6) 
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we had actually the equivalence 

P{n,p)^Tsin) (7) 

and 

Tp{n,p) = l-P{n,p) (8) 

Following Gustafson, a serial processor would require time Ts{n) + Tp{n,p) -p 
to perform the task, so the scaled speedup on the parallel system was equal: 

S{n,p) = ^i'^lJ^li^n^p^^ =Un) + T,{n^p).p = p+{l-p).Un) (9) 

Using the equivalence (JT]) we may write (l9|) in the following form: 

S{n,p) =p+{l-p)-Ts{n) =p+{l -p)-P{n,p) = p- {p-l) ■ (3{n,p) (10) 
The last equation is called Gustafson-Barsis law. 

3 The main results 

In the Gustafson's paper [2], three things raise some doubts: 

1. Mixing the problem size and the number of processors, treating both 
as tightly connected ("the problem size scales with the number of pro- 
cessors") 

2. Normalizing the time of calculations on the sequential machine to 1 
(eq. (jni)) for all problem sizes and numbers of processors 



3. Treating assessment ( iTOll as a better alternative to Amdahl's law, de- 
rived independently, basing on different assumptions 

The truth is, that Gustafson-Barsis law is nothing but a different form of 
Amdahl's law, and that better values of the speedup in the Gustafson's ex- 
periments with the growing size of the problem could be obtained directly 
from the Amdahl's law. 

To show this it is sufficient to notice, that for a given problem size 
n there is a constant in all executions of the program, on machines with 
different number of processors. This constant is the time of the execution of 
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the sequential part Ts{n) for the given problem size n. It is independent of 
the number of processors p, that is: 

Tgin) = (3{n,p) ■ T{n,p) = const., p = 1,2,3,... (11) 

So, it will be for any p = 1, 2, 3, . . . 

/3(n,l)-T(n,l) = /3(n,p)-r(n,p) (12) 

From the equation ( fT2l) we get: 

/3(n,l)=/3(n,p).^|^ (13) 

Replacing /3(n, 1) in equation ([3]) by (fT3ll we will get: 

T(n, p) = ,3(„, rt . T(n, p) + I<!^ - '^t"' ' (14) 

P P 

Now, multiplying both sides by p and moving all components with T{n,p) 
to the left hand side we receive: 

p ■ T{n,p) — p ■ P{n,p) ■ T{n,p) + (3{n,p) ■ T{n,p) = T{n, 1) (15) 

We will get the value of speedup S{n,p) obtained owing to the parallelization 
dividing both sides of the equation (fTSll by T{n,p). So, it will be equal: 

Sin,p) = =p-(p-l). p(n,p) (16) 

In this way we received nothing but Gustafson-Barsis law (fTOl) . 

What concerns the better speedup in Gustafson's experiments with the 
growing size of the problem (and the number of processors which was linked 
there) we may explain it in the following way. Gustafson assumed, that the 
time spent in the serial part ("for vector startup, program loading, serial 
bottlenecks, I/O operations") do not depend on the problem size, that is 

Ts{n) = const. =Ts = P{n,l) -Tin,!) = ps -Til,!), Vn (17) 

while the total time of the execution of the parallel part on the sequantial 
machine was proportional to the problem size n. In this way the serial factor 
on the sequential machine /3(n, 1) was equal 

Pin,!) 



(3s ■ T(l, 1) + n ■ (1 - /3,) • T(l, 1) p^ + n-{l-(3,) 1 + n ■ {j- 

(18) 



A similar situation would be when the time Ts{n) is proportional to the prob- 
lem size n (e.g. n ■ (3s), but the time spent in the parallel part is proportional 
to (e.g. ■ (1 — [3s)). In such cases 

(3{n,l) ^ (19) 

n ^ oo 

what means, taking into account dH), that 

S{n,p) p (20) 

n — » oo 

In other words, also from Amdahl's law we may conclude, that the bigger 
the size of the problem, the closer the speedup to the number of processors. 

4 Conclusions 

In the paper it is shown, that the Gustafson-Barsis law can be directly derived 
from the Amdahl's law, without strange assumptions as normalizng to one the 
time of execution of the program on the sequential machine. Moreover, the 
speedups approaching the number of processors observed in the experiments 
described in the Gustafson's paper can be concluded from the Amdahl's law, 
when we take into account as the arguments of the serial factor the size of 
the problem and the number of processors. 
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